Virtual decals for precision alignment and stabilization of motion graphics on mobile video

ABSTRACT

The virtual decals for precision alignment of motion graphics of the present invention provides a method by which a person using a mobile video capturing device may apply a special effect or motion graphic to the video being captured using a virtual decal present on the screen of the mobile device, providing a reference to the user allowing the user to compensate for camera movement and properly frame the video to which the motion graphic will be applied using post-processing. The present invention further provides a method for measuring the optic flow of a given video scene and applying that data in a predictive manner so as to stabilize a motion graphic against the scene. Post processing places such a motion graphic within the processed video in such a way that it appears to be part of the original scene.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a Utility patent application based on a previouslyfiled U.S. Provisional Patent application, U.S. Ser. No. 61/732,392filed on Dec. 2, 2012, the benefit of the filing date of which is herebyclaimed under 35 U.S.C. §119(e) and incorporated herein by reference.

TECHNICAL FIELD

The present invention relates generally to video recording andprocessing, and more particularly, but not exclusively to the videorecording and processing on mobile computers.

BACKGROUND OF THE INVENTION

The movie making industry has captivated the masses since the advent ofthe motion picture camera. The process of film making is constantlychanging and evolving as technology continues to advance. With theadvent of smart phones and charge-coupled device (“CCD”) image capturingtechnology, people are now able to create videos in any desired locationand time using their mobile phones and other mobile electronic devices.

Standard mobile phones or smartphones generally have the ability tocapture photos and videos, and often the ability to capture bothsimultaneously. Most recently, smartphones have begun offeringapplications or “apps” that further provide the ability to edit video onthe phone itself, without the aid of a personal computer. However, thereare no systems currently available allowing a user to select an effector motion graphic before actually filming a scene and apply it in such away that allows the user to properly frame the scene on the mobiledevice with a graphical representation of the effect or graphic.

Additionally, most video post-processing is conducted on highly capablepersonal computers, and most often large desktop machines with powerfulprocessors. Little has been done to allow effects to be added to videocaptured with a mobile device, right on that mobile device. Where amotion graphic or effect is applied to a scene, it most often moves withthe camera frame, as if it were fixed to the outside of the lens andlocated in a specific location in the video image. The graphic thenmoves where the camera “eye” moves, not fixed to the scene itself.

In light of the above, it would be advantageous to provide a method bywhich a user can select from a library of video effects within a videocapturing app on a mobile device, allowing the user to embed a resultingmotion graphic into a video without the use of a separate computer forvideo post-processing.

It would be further advantageous to provide a system that allows a userto apply a motion graphic to a video captured on a mobile device,applying a predictive algorithm to fix the graphic or effect against thescene, maintaining its position in the scene, as opposed to allowing thegraphic to move with the camera eye as it moves across the scene.

SUMMARY OF THE INVENTION

The present invention presents a method by which a user can create anaugmented video scene with embedded computer graphics and animation inmobile video capture through video post-processing.

The present invention provides the user with expanded information forflexibility in video creation and editing by allowing the user capturingvideo with a smartphone or similar mobile device to superimpose or embedgraphics into the video. The chosen graphics are represented as a“decal” on the user display screen, or camera viewfinder, of the mobiledevice while taking the video that is further movable within theconfines of the video. This virtual decal provides an image overlaydemonstrating where the graphics will occur on the processed video. Thisallows the user to frame the video by placing the chosen decal in thedesired location on the user display screen while filming.

Once the video is captured and stored in local media, post-processingembeds a motion graphic within the video, in the location indicated bythe position of the decal.

For instance, if a user wanted to embed a logo, copyright notification,animation, or some other image (“motion graphic”) within a video he orshe was shooting with a mobile device, the user would select the motiongraphic with a corresponding “decal” prior to shooting. During theshooting process, the decal appears in the user display screen of thecamera viewer, indicating the location where the final motion graphicwill appear in the processed video. When shooting is complete,post-processing embeds the motion graphic (represented by the decal)into the video, as if the decal were actually filmed in the video.

Additional embodiments of the present invention include the ability toinclude dynamic or animated virtual decals allowing a full effectsequence. In such an embodiment, the embedded motion graphics on themobile device's video remain stationary in circumstances in which thesource video incorporates movement of the camera.

The user selects a desired motion graphic at which point thecorresponding virtual decal appears in a specific location in the cameraviewfinder. The user is then able to identify the ultimate position ofthe motion graphic in the final processed video by the placement of thevirtual decal in the viewfinder, while recording the video. The motiongraphic remains in a stable position within the video, despite anymovements of the camera during the recording process. The inventionutilizes principles of “optic flow,” compensating for camera motion orpanning, in order to preserve the position of the motion graphicrelative to the motion in the video image. Once the user completesshooting, post-processing embeds the motion graphic associated with thevirtual decal in use within the final video, maintaining the motiongraphic's position relative to the video subject as the camera moves.The result is a video with a motion graphic that remains stationarywithin the dynamic space of the video, enabling the illusion of realism.

For instance, such an embodiment allows the user to embed a dynamiccomputer generated animation, such as a fire or moving character, intothe video. While the user may pan left, right, up, or down with thecamera, the position of the animation or graphic represented by thevirtual decal is preserved within the resulting video.

This further allows for the insertion of branded objects into effectsequences creating advertising opportunities within user-generatedmobile video.

DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments herein are described withreference to the following drawings. In the drawings, like referencenumerals refer to like parts throughout the various figures unlessotherwise specified.

FIG. 1 is a frame capture from a video-capture app utilizing a preferredembodiment of the present invention, showing user-selectable virtualdecals, mobile video, and options controls;

FIG. 2 is a flow chart of a preferred embodiment of the presentinvention showing the step-by-step process in which a mobile applicationusing virtual decals for precision alignment of motion graphics isinitiated and implemented by the user, as the user selects the desiredeffect from an effect database, initiates video capture with a mobiledevice, frames the video using the location of the virtual decal in thecamera viewfinder, records a video, processes the video, and ultimatelyviews the final video scene with the selected effect embedded within thevideo;

FIG. 3 is a flow chart of an alternative preferred embodiment of thepresent invention showing the process by which a computer-generatedmotion graphic is fixed in space within a mobile video by processing thevideo for recognizable and repeatable points within each frame, usingoptical flow allowing the process to locate each point in subsequentframe, predicting perspective changes of the inserted graphic,calculating movement of each point based on the radius of prior frames,translating the motion of the points into data, applying the motiongraphic to the video based on the translation data, offsetting theposition of the motion graphic to the video for image stabilization,examining frame-to-frame changes for error limitation, continuouslyevaluating and correcting for video errors, ultimately resulting in avideo with an embedded motion graphic that maintains its position in thevideo regardless of camera motion;

FIG. 4 is a representative example of a mobile electronic device havinga recording capability and capturing a video image containing alandscape, with the post-processed video being displayed on the videodisplay and having the embedded animation or graphic of a car positionedwithin the video;

FIG. 5A is a representative example of the present invention embodied ina mobile electronic device as a landscape is being viewed and having avirtual decal resembling a “thought bubble” positioned in the lower leftportion of the video display, or viewfinder, as the mobile electronicdevice is panned across the landscape;

FIG. 5B is a representative example of the mobile electronic deviceshown in FIG. 5A displaying the post-processed video which has beenenhanced with a motion graph in the location of the virtual decal, withthe motion graphic changes yet is maintained in the same position of thevideo display as the remainder of the video image pans across thelandscape

FIG. 6 is a representative block diagram of an exemplary mobileelectronic device embodying the present invention, and includes acentral processor which received image data from a Charge-Coupled-Device(CCD) Imaging Sensor that has been processed by a CCD Image Processor, amemory having an effect choice database, a memory having a virtual decalimage database, an output video storage memory, and having a userinterface with a display, and equipped with a radio frequencytransceiver or video output interface;

FIG. 7 is a representative example of the present invention embodied ina mobile electronic device as a landscape is being viewed through theviewfinder or display and showing a virtual decal corresponding to amotion graphic positioned on the screen providing visual guidance of thelocation of a to-be-applied motion graphic, and also contains an imageof the landscape about to be video recorded as the device is pannedacross the landscape as the items in the landscape move in nature, suchas the river flowing and the vehicle driving off;

FIG. 7A is a representative example of the device of FIG. 7 showing thepost-processed video containing a motion graphic resembling an incomingmeteorite heading in the direction of the previously displayed virtualdecal (shown in FIG. 7) and which trajectory remains stable relative tothe landscape shown in the video despite the movement of the device asit pans from left to right across the landscape;

FIG. 7B is a representative example of the device of FIG. 7 showing thepost-processed video containing the motion graphic now resembling ameteorite impacting the landscape in the location of the previouslydisplayed virtual decal (shown in FIG. 7) while the location of themotion graphic remains stable relative to the landscape shown in thevideo despite the movement of the device as it continues to pan fromleft to right across the landscape;

FIG. 7C is a representative example of the device of FIG. 7 showing thepost-processed video containing the motion graphic now resembling a fireresulting from a meteorite impacting the landscape in the location ofthe previously displayed virtual decal (shown in FIG. 7) while thelocation of the motion graphic remains stable relative to the landscapeshown in the video despite the movement of the device, and despitechanges of items within the field of view, such as the truck drivingaway from the now-burning meteorite;

FIGS. 8 and 8A-8C are exemplary views of the device shown in FIG. 4equipped with a recording capability and capturing a video imagecontaining a landscape, and showing a recording button having acount-down timer which provides a user with a graphical representationof the time remaining corresponding to a specific motion graphicselected, with the recording button having a stopwatch arm rotatingclockwise to zero with the portion representing the time used being of afirst color or pattern, and the portion representing the time remainingbeing of a second color or pattern;

FIG. 9 is an exemplary view of the device shown in FIG. 8 displaying thepost-processed video on the video display and having the embeddedanimation or graphic of a car positioned within the video, with a videotimeline representing the entire period of the video to be processed,and having a sliding bar within the video timeline corresponding to thetime period of the video graphic, which can be positioned along thetimeline to provide for the selective placement of the video graphwithin the original video such that the video graphic will appear withinthe post-processed video at a specific point in the video;

FIG. 9A is a photographic representation of the device of FIG. 9,showing an exemplary virtual decal on a display screen from a previouslyrecorded video, and a sliding time scale showing the timeline of thevideo, and the shorter timeline of the video graph able to be slidablypositioned within the video for post-processing;

FIG. 10A through 10C are a series of screen-shots from a video havingbeen created using the device of the present invention, including avideo field of view which changes, and which is post-processed toinclude a video graphic resembling a flamethrower discharging a largevolume of burning fuel towards a building; and

FIG. 11 illustrates maintaining multiple coordinates systems fortracking the motion graphic, in accordance with at least one of thevarious embodiments.

DETAILED DESCRIPTION

Various embodiments now will be described more fully hereinafter withreference to the accompanying drawings, which form a part hereof, andwhich show, by way of illustration, specific embodiments by which theinvention may be practiced. The embodiments may, however, be embodied inmany different forms and should not be construed as limited to theembodiments set forth herein; rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the embodiments to those skilled in the art. Amongother things, the various embodiments may be methods, systems, media, ordevices. Accordingly, the various embodiments may take the form of anentirely hardware embodiment, an entirely software embodiment, or anembodiment combining software and hardware aspects. The followingdetailed description is, therefore, not to be taken in a limiting sense.

Throughout the specification and claims, the following terms take themeanings explicitly associated herein, unless the context clearlydictates otherwise. The term “herein” refers to the specification,claims, and drawings associated with the current application. The phrase“in one embodiment” as used herein does not necessarily refer to thesame embodiment, though it may. Furthermore, the phrase “in anotherembodiment” as used herein does not necessarily refer to a differentembodiment, although it may. Thus, as described below, variousembodiments of the invention may be readily combined, without departingfrom the scope or spirit of the invention.

In addition, as used herein, the term “or” is an inclusive “or”operator, and is equivalent to the term “and/or,” unless the contextclearly dictates otherwise. The term “based on” is not exclusive andallows for being based on additional factors not described, unless thecontext clearly dictates otherwise. In addition, throughout thespecification, the meaning of “a,” “an,” and “the” include pluralreferences. The meaning of “in” includes “in” and “on.”

As used herein the term “motion graphic” may refer to a specific videoanimation and/or special effect graphic selectable from the motiongraphic and/or special effect library. A motion graphic may comprisevarious images, computer-generated graphics, animations, or the like,that can be embedded in a video.

As used herein the term “virtual decal” may refer to an image overlaydemonstrating where the motion graphics will appear on the processedvideo. During the video capture process, the virtual decal may appear inthe user display screen and/or view finder of a mobile computer and/orcamera viewer, indicating the location where the motion graphic willappear in the processed video.

As used herein the term “tracking points” may refer to several pointsdetermined within a video frame that may be identified from one videoframe to the next in a sequence of captured video frame. Sufficientnumber of points may be identified to enable tracking the location ofthe tracking points between video frames. In some embodiments, aconfiguration setting may be used for setting the minimum and/or maximumrange of the number points to be determined.

As used herein the term “reference point” may refer to one or morepoints or positions within a video frame that may be determined from thetracking points. Reference points may be employed for determining theprecise location within each video frame to use for locating andembedding the motion graphic. As tracking points change position fromvideo frame to frame to video the reference points for a video frame maybe updated accordingly. Thus, as the reference point changes location ina video frame, the location of where the motion graphic is embeddedwithin the video frame may be modified based in part on the change ofposition of the reference point.

As used herein the term “video scene” may refer to an area viewable in avideo recording view finder or it may refer to a recorded video of ascene. Video scenes may include image data captured from an area of viewthat exceeds the dimension of a single video frame or the area that isviewable at a given time in the video recorder's view finder. In somecases video scenes may have a definite start time and stop time, orotherwise have a set duration based on the amount of video that has beenrecorded. Virtual decals may be overlaid on a view finder during therecording of a video scene. Likewise, motion graphics may be embeddedinto a video scene after it has been recorded.

As used herein the term “video frame” may refer to captured image datarepresenting a single image from a sequence of images captured as partof recording a video scene. In some embodiments there may be 12, 24, 30,60, or more, video frames per second of recorded video depending on thecapability of the video recorder and/or configuration setting used whenrecoding the video.

As used herein the term “quality score” may refer to numeric value thatis assigned to each tracking point. The quality score may be employed todetermine if an individual tracking point should be kept or discarded.For example, in at least one of the various embodiments, a quality scoremay be computed based on information that includes determining if themotion of the tracking point is deviating from its predicted location.

Referring initially to FIG. 1, a screen capture of a mobile device videocapture application (“app”) is generally labeled 100. Screen capture 100comes from a video sequence 102 of a parking lot, 12 seconds into thevideo, as depicted by time stamp 104.

Also shown on the screen capture 100 of the app are a record button 106,a flash button 108, cancel button 110, mode selector 112, along withmotion decals 114 and 116, both depicted as outlines on the screencapture 100.

The user will initiate a video processing application, such as, videoprocessing application 678 and/or image tracking application 682 on amobile device (e.g., a mobile computer) and select the motion graphicfor inserting into the video. Once the motion graphic is selected theuser is presented with a camera viewfinder 100, which includes thevirtual decals 114 and 116 of the corresponding motion graphic. The usermay choose to turn the decals 114 and 116 on or off with mode selector112. The decals may be static or animated depending on the motiongraphic selected. Cancel button 110 closes the viewfinder and returnsthe user to a library of alternative motion graphics. Record button 106commences video capture by the mobile device. Time stamp 104 canindicate one of a number of possible events, such as the time remainingin a given animated motion graphic's animated sequence, or the elapsedtime recorded, providing the user with an indication of when the effectis complete and recording can cease. In an embodiment, options areavailable allowing the user to display multiple time stamps at once.

In a preferred embodiment, video sequence 102 is has a stationary cameraposition, viewing a single scene from a static angle. Virtual decals 114and 116 are associated with a specific motion graphic selectable fromthe effect library (not shown). The effect library is contains theimages, computer-generated graphics, and animations that can be embeddedin the video. The selected effect is ultimately incorporated into agiven video sequence 102 as a motion graphic. Once the user selects aneffect from the library, a virtual decal is available for display in thecamera viewfinder 100, indicating the ultimate position of the motiongraphic that will be embedded within the video sequence 102.

In at least one of the various embodiments, virtual decals arerepresented as static line drawings of the desired effect, reducing theamount of processing power required to represent them during videocapture. In an alternative embodiment, virtual decals may be representedon screen as fully animated versions of the resulting motion graphic,providing the user with a more accurate depiction of how the motiongraphic will appear within the resulting video scene.

Further, in at least one of the various embodiments, video processingapplication 678 may be arranged to enable the user to adjust the sizeand/or scale of the virtual decal, which may determine the size and/orscale of the how the motion graphic may be rendered into the final videoscene. In at least one of the various embodiments, a touch screeninterface may enable the user to adjust the position, size, orientation,or the like, or combination thereof, of the virtual decal directly inthe mobile computer's view finder. The user may then begin recording adesired video scene by activating record button 106, framing the videoscene with respect to static virtual decals 114 and 116, each embodyinga component of the motion graphic effect. While virtual decals 114 and116 are not incorporated into the video sequence 102 while recording,both virtual decals 114 and 116 provide the user with a reference as towhere the resulting motion graphic will be embedded once video sequence102 is recorded and processed.

In a preferred embodiment, virtual decals 114 and 116 need not be asingle image, but can be a computer-generated animation or graphicalsequence. Both virtual decals 114 and 116 can thereby move within thevideo sequence 102 as dictated by the computer-generated animation orgraphical sequence, allowing the user to adjust the camera angle asrequired.

For instance, as screen capture 100 shows, video sequence 102 containsvirtual decal 114, depicted as a robot figure with a gun and decal 116is depicted as a crosshair and explosion where the robot is aiming. Bothvirtual decals 114 and 116 remain referenced to a single spot within thevideo sequence 102, and reflect moments in the motion graphic for theuser to obtain perspective while recording video sequence 102.

In an embodiment, virtual decals 114 and 116 may be depicted as seriesof images as in an animation, such as a robot walking across the screencreating an explosion, further aided by a graphical vector line (notshown) describing the path of the animation across the screen for theuser's reference.

FIGS. 2 and 3 use flow charts to illustrate the general operations thatmay be performed by at least one of the various embodiments, includingby video processing application 678 and image tracking application 682.In at least one of the various embodiments, processes 200, and 300described in conjunction with FIGS. 2 and 3 may be implemented by and/orexecuted on a single mobile computer, such as mobile computer 650 ofFIG. 6. However, embodiments are not so limited and various combinationsof mobile computer, network computers, desktop computers, video camera,or the like may be utilized.

Referring now to FIG. 2, a block diagram of a preferred embodiment ofthe present invention is depicted and generally labeled process 200. Theuser will start at step 202 with the initiation of a particular mobilevideo capture device. At step 204, the user will launch a givenapplication that makes use of the virtual decals for precision alignmentof motion graphics of the present invention, and access the effectdatabase at step 206. In step 208, the mobile device will displayavailable effects, allowing the user to select, in step 210, a desiredmotion graphic that will then be represented onscreen as a static ordynamic virtual decal 114 and 116. In step 210, the library of effectscan be displayed on the mobile device in such a manner as to depict theassociated motion graphic as it would appear in a completed video,allowing the user to make an informed decision as to how to frame agiven video sequence 102.

In an alternative embodiment, through the use of options button 108, theuser will also be able to view the effect library in a manner that showsvirtual decals 114 and 116 as they would appear as motion graphics inthe video capture (step 222, below). Once selected, the user may alsoadjust the size of a given effect or virtual decal affecting theultimate size of the motion graphic in the resulting video sequence 102.In step 212, the process will retrieve the effect selected by the userin step 210, allowing the user to view the selected virtual decals 114and 116 in a video capture. User may activate or deactivate selectedvirtual decals from view with mode selector button 112. Step 214initiates a given mobile capture device, such as the embedded CCD camerain a common smartphone. It is to be appreciated by those skilled in theart that the use of a smartphone in the present example is not to beconsidered limiting. The use of other mobile devices such as a tabletpersonal computer (“tablet PC”) or other mobile video capture device donot depart from the spirit of the invention.

In step 216, a virtual decal, such as a static virtual decal 114 or adynamic virtual decal 116, corresponding to the effect choice isviewable within the viewfinder of the camera and video capture app onthe user's mobile device. The user may then size and position thevirtual decal within the frame of the video capture device in order toalign it with the desired subject, as stated in step 220. In step 222,the user begins recording a desired video sequence 102, while remainingable to position the virtual decal in the field of view as required.

Method steps 226 through 230 are related to a visual timing cue that mayprovide a user with feedback regarding the minimum video durationrequired for a particular video graphic, and will be discussed furtherin conjunction with FIGS. 8 and 8 A-C.

In an embodiment, the user may select a virtual decal 114 in order toplace a cartoon image or other graphic sequence or animation in theframe, as shown in FIG. 1. Alternatively, in the case of anadvertisement or copyright notice, the resulting motion graphic will bedisplayed in a in a position within the field of view of the frame,independent of the camera's angle. Step 228 in process 200 is the systemfunction that maintains the position of the virtual decal 114 or 116 inthe user-selected position as the user captures desired footage. Thisstep 228 ensures that the reference for the virtual decal 114 or 116 ismaintained throughout filming.

Once the video capture is complete, the user may end recording with hismobile video capture device in step 234, and proceed to step 236, inwhich the mobile device's video editing application will process thevideo to embed the motion graphic represented by the virtual decal(e.g., virtual decals 114 and 116) at the appropriate position in theappropriate frames of the video sequence 102. Then the user may beenabled to view the resulting video sequence 102 with embedded graphicsor animations in a final video scene in step 238. Once the process hasended in step 240, the user may then further edit or share thepost-processed video containing the video-graphic as desired.

Accordingly, in at least one of the various embodiments, the same mobiledevice that captures the video may be the same mobile device thatperforms the image processing for tracking the graphic location andembedding the graphic into the recorded video.

Referring now to FIG. 3, an alternative preferred embodiment of thepresent invention is depicted as a flow chart, generally referred to asprocess 300. In at least one of the various embodiments, process 300 mayalso be understood as an expansion of step 236, picking up the processin the video processing stage, following video capture and allows theapplication of video motion tracking to mobile video.

Whereas process 200 provides a system for the user to place virtualdecal 114 or 116 in a position providing a reference for where themotion graphic will ultimately lie in the final video, process 300provides a method by which the motion graphic is referenced to thesubject of the video and maintains its position relative to the subjectof the video as the camera moves, pans, or tilts.

Process 300 starts 302 with step 304, wherein the motion graphicassociated with the virtual decal 114 or 116 chosen from the effectlibrary in step 210 may be processed into individual frames of mobilevideo feeds at any frame rate or resolution to account for variancebetween mobile device capabilities. In at least one of the variousembodiments, process 300 may be arranged to process each frame of videosequentially.

Within each video frame, process 300 identifies several points on eachframe that can be recognized as varying from one frame to the next basedon a number of properties including contrast, lighting, shape, shadow,size, etc. These tracking points are, more particularly, specific pixelsor areas of pixels within the image of each frame. Each of thesetracking points within each video frame is referenced to the adjacentframes and used to track the motion of the camera. Also, points on eachframe may be utilized to determine the optic flow of the image as thecamera is panned or moved.

Step 306 multiplies the process in step 304, as the system of process300 selects between forty and eighty points per frame that may be themost effectively identified from one video frame to the next. In atleast one of the various embodiments, more or less points per frame maybe determined such that a sufficient number of points may be identifiedto enable tracking between frames. In some embodiments, a configurationsetting may be used for setting the minimum and/or maximum range of thenumber points to be determined. These points may be referred to astracking points.

In at least one of the various embodiments, the determined trackingpoints may be employed to establish one or more reference points thatmay be used for determining the precise location to use for locating andembedding the motion graphic within each video frame. If the referencepoint changes locations in subsequent video frames the location of themotion graphic may be offset accordingly. For example, in at least oneof the various embodiments, assuming a motion graphic that is located at(100, 100) in a coordinate system and the reference point based on thetracking points is at (200,100), if inter-frame camera shake moves thereference point to (205, 100), the motion graphic location may be offsetto (105,100) to account for the motion contributed by the camera shake.This compensation makes the motion graphic appear to be stationary withthe video image even though the camera may have moved. Likewise, themotion graphic location may be offset to account for changes in thereference point location based on deliberate camera movements, such aspanning, zooming, tilting, or the like.

In at least one of the various embodiments, a tracking point may bedetermined based on computing a score based on a combination ofdifferent factors, such as, contrast, brightness, darkness, patternmatching, edge detection, corner-recognition, or the like. In at leastone of the various embodiments, one or more of the factors may beweighted based on their strength of the correlation with identifyingquality points suitable for tracking between frames. In at least one ofthe various embodiments, the weighting values may be determined and/ormodified using configuration settings. In a preferred embodiment, thisprocess is adaptive over time and depends on the composition of thevideo, or more specifically contrast features, shapes, or even colors.In an alternative embodiment, user-selectable options allow the user toadjust the number of enabled reference points and where they arepositioned.

In at least one of the various embodiments, video frames may be analyzedto determine one or more characteristics of the video, such as,brightness, contrasts, motion, colors, indoors, outdoors, or the like.The factors used for determining the tracking points may be modifiedbased on one or more of these characteristics. For example, in at leastone of the various embodiments, in low contrast video, edge detectionmay be difficult and/or unreliable while point brightness may bereliable. Accordingly, in this example, the weight value for brightnessvalues may be increased while the weight value for edges detection maybe reduced.

In at least one of the various embodiments, one or more formulas may beemployed for computer score for points and/or areas of points that maybe included in a video frame, including:

${f(A)} = {\sum\limits_{p \in A}^{\;}\; ^{- {(\frac{{I{(p)}} - {I{(p_{0})}}}{t})}^{q}}}$

Where, in at least one of the various embodiments, the function fdetermines the quality score of an area A centered around a point p0,where I represents the intensity of a point p, and t and q aredetermined empirically. Here, t and q may be weighting factors thatcorrespond to basing at least a portion of the quality score of theillumination factor of the pixels that correspond to each point. In atleast one of the various embodiments, other factors may be included,and/or substituted as well, including, contrast, brightness, darkness,pattern matching, edge detection, corner-recognition, or the like.

Step 308 determines the points for enabling the system to later identifymotion and/or scene changes between the images from frame to frame ofthe video in step 310. For example, this step 308 can selecthigh-contrast areas in individual video frames or sharp edges and shapesin the features of the captured video, as tracking points. In at leastone of the various embodiments, the tracking points may be identifiedbased on quality scores computed based on the pixels in the frame thatare associated with the points.

Step 310 then tracks the most identifiable spatially distributedtracking points (from step 308) from frame to frame and compares theirposition. As the tracking points vary from frame to frame, step 312examines the optical flow of individual tracking points. Optical flow inthis sense refers to the pattern of apparent motion of objects,surfaces, and edges in a visual scene caused by the relative motionbetween an observer and the scene. In step 312 this refers specificallyto the relative motion of the scene being filmed and the mobile videocapture device, as well as the scene's directional movement, rotation,zoom, and any other warping of the scene as viewed by the camera.

The data determined in step 312, may be compared from frame to frame andthe differences in the position of the individual tracking points may beused in a predictive manner in step 314 to estimate perspective andposition changes in the position of the tracking points from frame toframe.

Step 316 calculates the most likely movement of each tracking pointbased on the radius of travel from the prior frame or frames. Thisprocess reduces the likelihood of statistical error and resulting videoerror. In at least one of the various embodiments, this may enabletracking points that may be moving and/or accelerating inconsistently ascompared with other tracking points to be determined and discarded fromthe set of tracking points. For example, in at least one of the variousembodiments, if a first portion of the tracking points correspond to abuilding or other fixed structure in the view field, and a secondportion of the tracking points correspond to a stationary automobileeach portion of tracking points should remain within an expected radiusin subsequent video frames. However, in this example, if the automobilebegins to drive away, in subsequent video frames, the tracking pointscorresponding to the automobile will be begin to move beyond the radiusdefined for their expected location.

In at least one of the various embodiments, a quality score for eachtracking point may be modified based on the variance of the predictedlocation of the tracking point within the video frame and the actualposition. For example, if a tracking point determined in a first videoframe is located close to a predicted location in a subsequent videoframe, the quality score for that point may be determined to be higherthan if the tracking point in the subsequent video had drifted to anunexpected location. Note, the tracking point may be expected to driftin accordance with the determined optical flow for the video scene.Deviating away from an expected location in the video frame reduces thequality score of the tracking point because its location is not reliablypredicted.

Step 318 translates the data from step 316, enabling application of thedata to adjust the position of the motion graphic selected by the user.In a preferred embodiment, the location of the effect, or motion graphicwithin the resulting video may be annotated by the use of a virtualdecal 114 or 116, as in process 200; however, a virtual decal 114 or 116is not required to affect process 300. In step 320, the selected motiongraphic is applied to the video frames, either individually, or in bulk,depending on the frame rate of the captured video and the rate of changeof motion graphic. The position of the motion graphic will be predicatedon whether it is a single graphic or a group or sequence ofcomputer-generated images or animations.

Once the motion graphic is applied to the individual video frames, theposition of the motion graphic is adjusted for optic flow, in step 322.The data generated in step 316 and translated in 318 is processed andinversely applied to the position of the motion graphic toproportionately offset aspect or position shifts between video frames,stabilizing the graphic against any movement between frames of thevideo. This maintains a “fixed” position of the motion graphic in theresulting video, as if the motion graphic was actually part of theoriginal scene.

In at least one of the various embodiments, the position of the trackingpoints may be used to translate the position of the graphic tocompensate for translation (motion) accountable to the motion of thecamera.

Step 324 utilizes an algorithm to analyze the movement of trackingpoints from one frame to the next, and calculates the likely futurelocation of each tracking point within each video frame. In at least oneof the various embodiments, one or more well-known image processingalgorithms may be employed as part of determining optical flow andmotion data. In at least one of the various embodiments, one or morewell-known gradient based algorithms, such as, least-squares, leastmedian of square error, or the like, may be employed to determine opticflow and/or motion data for the video frame. Likewise, in at least oneof the various embodiments, one or more well known statistical basedalgorithms, such as, RANSAC (RANdom SAmple Consensus), or the like, maybe employed. One of ordinary skill in the art will appreciate that otheralgorithms for determining optic flow and/or motion data for videoframes may be used without departing from the scope of the innovationsdisclosed herein. In at least one of the various embodiments,information provided from one or more motion sensors such as thosedescribed in conjunction with FIG. 3 (e.g., gyroscopes, accelerometers,or the like) may be used if determining optical flow and/or motion data.The information from the physical sensors on the mobile computer may beemployed to determine motion/movement in the video frames that may havebeen introduced by movement of the mobile computer employed to record tothe video frames. In at least one of the various embodiments, some orall of the motion sensor information may be used to confirm and/orcorrect the prediction computations used for determining optical flowand/or motion data for a video frame.

If the determined movement of a tracking point significantly differsfrom its predicted movement and/or the motion data for that point startsto significantly diverge from the optic flow and/or motion data in thescene, the system may determine a disruptive event has occurred. Forexample, determined tracking points may be associated with objects inthe scene that begin to move independent of camera movement. As theseobjects move within the scene, the associated tracking point locationswill begin to diverge from their expected path and/or location. Step 326provides for continuous evaluation of the selected tracking points forcalculation errors, providing real-time error notification andcorrection. If the preset maximum error rate is exceeded, step 328 willloop back to step 306 (indicated by connector A), wherein the process300 may dynamically determine new tracking points for calculation of theoptic flow for referencing the motion graphic with the camera motion. Iftracking points are determined to be varying and/or changing beyond thesystem's capacity to continue using them for tracking they may bediscarded and new tracking points may be determined. In at least one ofthe various embodiments, the new tracking points may be determined suchthat they may have similar characteristics to the tracking points thatwere discarded to preserve the integrity of the tracking. In anembodiment, error rates and/or variance thresholds are fixed within thecode of the program, however in alternative embodiments, error ratesand/or variance thresholds may be adaptive or user-selectable in a givenapplication, using direct user inputs and/or configuration settings.

In at least one of the various embodiments, each tracking point may beevaluated for quality. In at least one of the various embodiments,quality may be based on the age of the tracking point and the variancebetween its predicted location and its actual location. In at least oneof the various embodiments, the age of a tracking point may be computedbased on the number of video frames that have been processed since thetracking point was first identified as a tracking point.

In at least one of the various embodiments, if the quality score for atracking point drops below a defined threshold, the tracking point maybe discarded from the set of tracking points. Accordingly, in at leastone of the various embodiments, if the number of current tracking pointsdrops below a defined threshold, the process may loop back to step 306to determine additional tracking points.

In at least one of the various embodiments, in cases where the number oftracking points drops below the defined threshold, the remainingtracking points may have high quality values. Accordingly, such trackingpoints may be automatically included in the set of “new” trackingpoints. For example, if 15 is defined as the minimum number of trackingpoints, process 300 may proceed to step 306 to acquire more trackingpoints when the number tracking points drops below 15. However, for thisexample, the remaining tracking points may be automatically included inthe new set of tracking points rather than being discarded.

Alternatively, if the error rate is not exceeded in step 328, step 330allows the user to view the final video scene with the embedded motiongraphic that is positioned in the video frame based on referenceinformation derived from the tracking points.

In at least one of the various embodiments, process 300 may be arrangedto maintain at least two Cartesian coordinate systems. One coordinatesystem, the motion coordinate system, may represent an area larger thanthe video frame and another coordinate system, the frame coordinatesystem may represent the actual area of the video frame.

In at least one of the various embodiments, the motion coordinate systemmay be used for tracking the location of motion graphic duringprocessing. Since the motion coordinate system may cover more area thanthe frame coordinate system, the process may be enabled to track thelocation of the motion graphic even if it is out of the viewing area ofthe video frame (e.g., not visible in the video frame. For example, ifthe motion graphic comprises a flying saucer, flying into the on screenview of the scene from the left and exiting to the right of the scene,the motion coordinate system may be used to track the location of theflying saucer motion graphic while it is out of view (out of frame).Accordingly, in at least one of the various embodiments, process 300 maybe arranged to translate the location of the motion graphic from themotion coordinate system to the frame coordinate system. See, FIG. 11.

After the end of this process 300, at step 332, the user may thenfurther process the resulting video or share it at will.

ILLUSTRATIVE EMBODIMENTS

Referring now to FIG. 4, a representative example of a mobile electronicdevice embodying the present invention is shown adjacent a field of view400 containing landscape 402. Specifically, a mobile electronic device404 includes a display 406 which is often also referred to as aviewfinder when the device is in the video capture mode. Display 406 maybe equipped with a “recording” indicator 408 which, in some embodiments,may blink red as is customary in the industry. A chronometer, or timecode, 412 may also be included in the display 406. Furthermore,recording indicator 408 may embody a countdown timer within theindicator, reflecting the duration of time recorded without requiring aview of the time code in 412.

In this example, device 404 has recorded a video within the field ofview 400 and including landscape 402. The recorded video has beenpost-processed to include a motion graphic 410, which in this caseresembles a vehicle such that the post-processed video now depicts thevehicle 410 being present in the landscape 402. Since this example isrepresentative of the post-processed video, the previously shown virtualdecal is not shown as it was replaced with vehicle motion graphic 410through the processing of the recorded video.

Referring now to FIG. 5A, a representative example of the presentinvention embodied in a mobile electronic device is shown in a field ofview 500 which includes a landscape 502 with moving elements, such astruck 504 moving in direction 506, and river 508. A mobile electronicdevice 520 includes a display 522, or viewfinder if in the recordingmode, and having a recording indicator 526, reflecting through acountdown timer the same elapsed recording time as 524. As shown,viewfinder 522 is showing an image 528 of a portion of landscape 502 infield of view 500. Additionally, a virtual decal 530 resembling a“thought bubble” has been selected and positioned in the lower leftportion of the video display 522, or viewfinder, as the mobileelectronic device is panned across the landscape in direction 532.

It is to be understood that virtual decal 530 corresponds to apreviously selected motion graphic and presented in a fashion thatgraphically identifies it as a virtual decal. For instance, virtualdecal 530 can be represented by lines or shapes which are colored,dashed, etc. to distinguish the virtual decal 530 from the landscapeimage 528. In some cases, as in the “thought bubble” in FIG. 5A, theshape of virtual decal 530 resembles the motion graphic; however, thisis not a requirement for the present invention.

FIG. 5B is another representative example of the mobile device shown inFIG. 5A displaying the post-processed video 528 which has been enhancedwith a motion graphic 534 in the prior location of the virtual decal 530on display 522. The motion graphic 534 changes in accordance with themotion graphic selected, yet is maintained in the same position of thevideo display 522 as the remainder of the video image 528 pans acrossthe landscape.

Referring now to FIG. 6, a representative block diagram of an exemplarymobile computer embodying the present invention is shown and generallydesignated mobile computer 650. Mobile computer 650 may be employed as amobile device for performing and/or enabling some or all of the actionsdescribed herein. Mobile Computer 650 includes a processor 652, such asa digital data processor, microprocessor, microcontroller, orprogrammable logic device known in the art and capable of completing thefunctions described herein. Central processor 622 receives image datafrom a Charge-Coupled-Device (CCD) Imaging Sensor 654 that has beenprocessed by a CCD Image Processor 656. A memory 658 having an effectchoice database provides a memory storage for video effects from which auser can select a desired motion graphic, which is provided directly tothe processor 652. A memory 660 having a virtual decal image databaseprovides a memory storage for virtual decals corresponding to the motiongraphic. Also, an source video storage 662 and an output video storage664 provides memory storage locations for video prior to incorporatingthe motion graphic, and post-processed once the motion graphic has beenapplied.

While device 650 has been shown to include multiple memory devicesincluding effect choice database 658, virtual decal image database 660,source video storage 662, and output video storage 664, it is to beappreciated that the various memory can be combined into single memorydevices having sufficient storage capacity and appropriate fileindexing. Indeed, all memory functions may be incorporated intoprocessor 652 without departing from the present invention.

Mobile Computer 650 may also include a mobile display device 666, suchas a liquid crystal display (LCD) or equivalent known in the art, whichserves as a viewfinder and video display, and may also include a userinterface 668 which may include a keyboard, trackball, pressure orelectrostatic sensitive membrane to perceive input from a user. MobileComputer 650 may also include a radio frequency transceiver 670 or videooutput interface 674 to provide input to the device from otherelectronic devices, or to transmit post-processed video to other devicesfrom antenna 672.

A timer 680 is provided to maintain timing information for the videographic selected for use in the device of the present invention. Timer680 may be programmed with a time entry corresponding to the effectchoice selected from database 658, and provide the processor withupdates once recording has begun to provide the user updates to minimumrecording times necessary to fully embed the effect choice and videographic in the recorded video.

In at least one of the various embodiments, although not shown, one ormore motion sensors, such as, gyroscopes, accelerometers, may beincluded as part of mobile computer 650 for measuring and/or maintainingan orientation of mobile computer 650, and/or measuring the magnitudeand/or direction of the acceleration of mobile computer 650.

Applications 676 may include computer executable instructions which,when executed by Mobile Computer 650, transmit, receive, and/orotherwise process instructions and data. Applications 676 may include,for example, Video Processing Application 678, and Image TrackingApplication 682. In at least one of the various embodiments, VideoProcessing Application 678 may be arranged to process video that may becaptured by CCD Imaging Sensor 654, including, embedding special effectgraphics, embedding animations, displaying virtual decals, or the like.Image Tracking Application 682 may be arranged to determined trackingpoints and track there position between video frames.

Other examples of application programs that may be operative on MobileComputer 650, include calendars, search programs, email clientapplications, IM applications, SMS applications, Voice Over InternetProtocol (VOIP) applications, contact managers, task managers,transcoders, database programs, word processing programs, securityapplications, spreadsheet programs, games, search programs, and soforth.

Referring now to FIG. 7, a representative example of an alternativeembodiment of the present invention embodied in a mobile electronicdevice is shown. A field of view 700 includes a landscape 702, as wellmoving elements, such as truck 704 moving in direction 706. Device 720includes a viewfinder or display 722 having a chronometer 724, and arecording indicator 726.

In a recording mode as a landscape 702 is being viewed through theviewfinder or display 722, a virtual decal 734 corresponding to a motiongraphic is positioned on the screen 722 providing visual guidance of thelocation of a to-be-applied motion graphic, and also contains an image728 of the landscape being video recorded as the device is panned acrossthe landscape in direction 732 as the items in the landscape move innature, such as the river 708 flowing and the vehicle 704 driving off indirection 706. The virtual decal 734 corresponding to the selectedmotion graphic is fixed in the image 728 displayed on the viewfinder722, and not fixed to a position on the viewfinder 722. Thus, when thedevice 720 pans across the landscape 702, the virtual decal 734maintains its position in the video and translates across the display722 corresponding to the translation of the image 728.

Button 726 initiates the recording of the video, and incorporates acountdown timer reflecting the duration of the motion graphic effectsequence. The video recorded is stored in memory and post-processed toinsert the appropriate selected motion graphic into the videocorresponding to the location of graphic 734.

FIG. 7A is a representative example of the device of FIG. 7 showing aninitial snapshot of the post-processed video containing a motion graphic730 resembling an incoming meteorite heading toward the location 734′ ofthe previously displayed virtual decal (not shown in this figure as itis not displayed in post-processed video). As the device 720 was pannedduring the recording of the video, the trajectory of the motion graphic730 remains stable relative to the landscape 702 shown in the videodespite the movement of the device 720 as it pans from left to rightacross the landscape.

FIG. 7B is a representative example of the device of FIG. 7 showing asubsequent snapshot of the post-processed video containing the motiongraphic 732 now resembling a meteorite impacting the landscape in thelocation 734′ of the previously displayed virtual decal 734 (shown inFIG. 7) while the location of the motion graphic 732 remains stablerelative to the landscape 702 shown in the video despite the movement ofthe device 720 as it continues to pan from left to right across thelandscape.

FIG. 7C is a representative example of the device of FIG. 7 showing thepost-processed video containing the motion graphic 732 now resembling afire resulting from a meteorite impacting the landscape 702 in thelocation 734′ of the previously displayed virtual decal 734 (shown inFIG. 7). Again, the location 734′ of the motion graphic 732 remainsstable relative to the landscape 702 shown in the video despite themovement of the device 720, and despite changes of items within thefield of view, such as the truck 704″ driving away from the now-burningmeteorite.

Referring now to FIG. 8 an exemplary view of an alternative embodimentof the device shown in FIG. 4 is generally designated 800. Device 800 isequipped with a display 802 and a video timer 804. The device isequipped with a recording capability and is capturing a video imagecontaining a landscape. A virtual decal 806 is positioned in display 802and represents the future position of a video graphic within the display802 following post-processing of the captured video.

A recording button 808 includes an integral count-down timer. Morespecifically, referring to the enlarged view of the recording button808, a graphical representation of the time remaining corresponding to aspecific motion graphic selected is provided. At the beginning of arecording, timer 808 is entirely a first color 810. Once the recordingbegins, stopwatch arm 812 rotates in a clockwise direction 814 to zerowith the portion representing the time used being of a first color orpattern 810, and the portion representing the time remaining being of asecond color or pattern 816. A transition period 818 may be provided togive the user a warning of the upcoming end of the video graphic.

In order for the device and method of the present invention to performoptimally, any video recorded must at least last the duration of themotion graphic being embedded therein. To provide the user with a cuewhen the video is long enough, we created a countdown inside therecording button. The button is fully lit when recording is initiated,and gradually dims in a clockwise manner as the recording time elapses.When the video duration is long enough to incorporate the graphic, thebutton is fully dimmed. When the user presses the button, the recordingstops and the button goes dark.

Another tool to assist a user in determining when a video is long enoughto insert a video graphic is implemented in the virtual decal 806itself. A visual cue notifies the user when sufficient video has beenrecorded to insert a specific video graphic. For instance, in oneembodiment of the cue, virtual decal 806 is initially a first color. Thecolor of the virtual decal changes from a first color, such as red, to asecond color, such as green, when the elapsed video time is long enoughto incorporate the motion graphic, thus making sure the video is longenough. This is particularly beneficial in circumstances when the useris focusing on the filming of a scene and not monitoring at the timedirectly.

Referring to FIGS. 8A, 8B and 8C, the implementation of the featuresdescribed are shown. In FIG. 8A, the recording button is completely red,a single color showing that the videotaping has not begun. Next in FIG.8B, as the videotaping proceeds, the stopwatch arm rotates changing thepreviously red button, to a slightly lighter red color to signify thetime period expired. Finally, in FIG. 8C, as the stopwatch arm hasrotated completely, the ring of the recorder button is completely thesecond, lighter, color indicating that the minimum period for the videographic has been achieved, and that the videotaping can now be stoppedif desired. Also, in FIG. 8C, at the completion of the minimum periodfor the video graphic, the virtual decal has changed to a second color,such as green, to indicate that the minimum recording period has beenachieved.

Referring now back to FIG. 2, method steps 226 through 230 are relatedto a visual timing cue that provides a user with feedback regarding theminimum video duration required for a particular video graphic. In step226, a countdown timer is initialized to provide a timer such that thevideo being taken will be long enough to insert the chosen videographic. During the recording, the virtual decal is maintained in thevideo scene in step 228.

If the timer has expired in step 230, the video recording may be endedin step 234, however, if the timer has not expired, return path 231leads to step 232 in which the timer is decremented and the virtualdecal timer effect is updated in step 233. The process returns to step228 where the virtual decal is maintained, and again the timer ischecked for expiration in step 230. This process continues until thetimer reaches zero, and continues to update the virtual decal timereffect as described above to provide the user with information regardingthe length of time the video must continue in order to make sure thatthe video graphic will fit within the post-processed video.

FIG. 9 is an exemplary view of the device shown in FIG. 8, and generallydesignated 900. Device 900 includes a display 902 and a timer 904. ThisFigure depicts the device 900 displaying the post-processed video on thevideo display 902 and having the embedded animation or graphic of a car905 positioned within the video, with a video timeline 906 representingthe entire period 910 of the video to be processed. A sliding bar 914within the video timeline 906 corresponds to the time period 916 of thevideo graphic. The sliding bar 914 may be positioned in direction 918along the timeline to provide for the selective placement of the videograph within the original video such that the video graphic will appearwithin the post-processed video at a specific point in the video.

Implementation of the above feature is depicted in FIG. 9A. In thisFigure, a video is being prepared for post-processing, and the placementof the sliding bar within the timeline provides for insertion of thevideo graphic at precisely the desired moment in the post-processedvideo.

Referring now to FIGS. 10A through 10C, a series of screen-shots from avideo having been created using the device of the present invention areshown. FIG. 10A includes a video field of view which changes as thevideo recording device is moved while recording. A video was recordedand post-processed to include a video graphic resembling a flamethrowerdischarging a large volume of burning fuel towards a building in FIG.10B. In FIG. 10C, the scene nears completion as the flamethrower isremoved, and the remaining burning fuel continues in the scene.

It is to be appreciated from these Figures that a rather mundane videotape showing the outside of a building may be augmented using thepresent invention to include a video graphic which provides an artisticand creative element to the video. It is fully contemplated in thisinvention that the video graphic can depict virtually anything, frommoving vehicles, flamethrowers, to wandering wildlife, alien life formsand whatever else can be imagined.

FIG. 11 illustrates system 1100 for maintaining multiple coordinatesystems for tracking the motion graphic, in accordance with at least oneof the various embodiments. As discussed above, image trackingapplication 682 may be arranged to include at least two coordinatesystems. Y-axis 1102 and x-axis 1104 may be the y and x axis for amotion coordinate system that may be larger than image frame 1106.Motion graphic 1108 may be determined to be progressing through thescene along tracking path 1110. Further, origin 1112 may be the originof the motion coordinate system.

In at least one of the various embodiments, a frame coordinate systembased on image frame 1106 may have an origin represented by origin 1114.Accordingly, x-axis for the frame coordinate system may be the lowerboundary of image frame 1106 extending from origin 1114 to the right.And, the y-axis for the frame coordinate system may be left boundary ofimage 1106 extending from origin 1114 upwards.

In at least one of the various embodiments, motion graphic 1108 may betracked in terms of the motion coordinate system (e.g., using pointsdefined by x-axis 1104 and y-axis 1102). If video processing application678 embeds motion graphic 1108 into image frame 1106 for the finalscene, the location of pixels/vectors comprising motion graphic 1108 maybe scaled, translated, and otherwise transformed from the motioncoordinate system to the frame coordinate system for image frame 1106.

In at least one of the various embodiments, since at certain timesmotion graphic 1108 may be entirely or partially out of the image frameand thus outside the bounds of the frame coordinate system, portions ofthe motion graphic determined to be outside of the frame coordinatesystem may be excluded from the processing required to embed the motiongraphic into the final scene. It will be understood that each blockand/or step of the flowchart illustrations, and combinations of blocksand/or steps in the flowchart illustrations, can be implemented bycomputer program instructions. These program instructions may beprovided to a processor to produce a machine, such that theinstructions, which execute on the processor, create means forimplementing the actions specified in the flowchart block/step orblocks/steps. The computer program instructions may be executed by aprocessor to cause a series of operational steps to be performed by theprocessor to produce a computer-implemented process such that theinstructions, which execute on the processor to provide operations forimplementing the actions specified in the flowchart block/step orblocks/steps. The computer program instructions may also cause at leastsome of the operational steps shown in the blocks of the flowchart to beperformed in parallel. Moreover, some of the steps may also be performedacross more than one processor, such as might arise in a multi-processorcomputer system. In addition, one or more blocks or combinations ofblocks in the flowchart illustration may also be performed concurrentlywith other blocks or combinations of blocks, or even in a differentsequence than illustrated without departing from the scope or spirit ofthe invention.

Accordingly, blocks/steps of the flowchart illustration supportcombinations of means for performing the specified actions, combinationsof steps for performing the specified actions and program instructionmeans for performing the specified actions. It will also be understoodthat each block of the flowchart illustration, and combinations ofblocks in the flowchart illustration, can be implemented by specialpurpose hardware-based systems, which perform the specified actions orsteps, or combinations of special purpose hardware and computerinstructions. The foregoing example should not be construed as limitingand/or exhaustive, but rather, an illustrative use case to show animplementation of at least one of the various embodiments of theinvention.

What is claimed as new and desired to be protected by Letters Patent ofthe United States is:
 1. A method for video processing using a mobilecomputer, comprising: selecting a motion graphic for displaying at alocation in a video scene; employing a view finder provided by themobile computer to view the video scene while positioning a virtualdecal at the location in the video scene, wherein the virtual decalrepresents the selected motion graphic; employing the mobile computer tocapture a sequence of video frames of the video scene while at least aportion of the virtual decal is viewable in the video scene with theview finder; and iteratively processing each recorded video frame fromthe sequence of captured video frames using the mobile computer, whereinthe processing includes: determining a plurality of tracking pointswithin each video frame based on at least one characteristic of an imageincluded in the video frame; determining a quality score for each of thetracking points; if the quality score corresponding to an individualtracking point included in the plurality of tracking points is less thana defined threshold, discarding the individual tracking point from theplurality of tracking points; determining at least one reference pointwithin the video frame based on at least the location of a portion ofthe plurality of determined tracking points; and embedding the motiongraphic in the video frame at the location of the virtual decal, whereinthe motion graphic is further positioned in the video frame based atleast on the reference point.
 2. The method of claim 1, furthercomprising, if a quantity of tracking points included in the pluralityof tracking points drops below a threshold, determine another pluralityof tracking points from a next video frame.
 3. The method of claim 1,wherein determining the quality score for each of the tracking points,further comprises, determining an age of the tracking point and thevariance between its predicted location and its actual location withineach video frame.
 4. The method of claim 1, wherein capturing thesequence of video frames of the video scene using the mobile computer,further comprises, displaying a visual timing cue in the mobilecomputer's view finder that indicates a duration for playing back themotion graphic.
 5. The method of claim 1, wherein displaying the virtualdecal in the video scene, further comprises, modifying a location of thevirtual decal within the view finder based on at least one dynamicproperty of the motion graphic.
 6. The method of claim 1, furthercomprising, enabling a user to adjust a size of the virtual decal fordetermining a size of the motion graphic for embedding in each of thevideo frames.
 7. The method claim 1, wherein determining a plurality oftracking points within the video frame, further comprises, computing ascore based on at least one of contrast, brightness, darkness, patternmatching, edge detection, or corner-recognition.
 8. The method of claim1, further comprising, plotting a position of the motion graphic using acoordinate system that is separate from another coordinate system thatcorresponds to each video frame.
 9. A mobile computer for videoprocessing, comprising: a transceiver for communicating over thenetwork; a memory for storing at least instructions; a processor devicethat is operative to execute instructions that enable actions,including: selecting a motion graphic for displaying at a location in avideo scene; employing a view finder provided by the mobile computer toview the video scene while positioning a virtual decal at the locationin the video scene, wherein the virtual decal represents the selectedmotion graphic; employing the mobile computer to capture a sequence ofvideo frames of the video scene while at least a portion of the virtualdecal is viewable in the video scene with the view finder; anditeratively processing each recorded video frame from the sequence ofcaptured video frames using the mobile computer, wherein the processingincludes: determining a plurality of tracking points within each videoframe based on at least one characteristic of an image included in thevideo frame; determining a quality score for each of the trackingpoints; if the quality score corresponding to an individual trackingpoint included in the plurality of tracking points is less than adefined threshold, discarding the individual tracking point from theplurality of tracking points; determining at least one reference pointwithin the video frame based on at least the location of a portion ofthe plurality of determined tracking points; and embedding the motiongraphic in the video frame at the location of the virtual decal, whereinthe motion graphic is further positioned in the video frame based atleast on the reference point.
 10. The mobile computer of claim 9,further comprising, if a quantity of tracking points included in theplurality of tracking points drops below a threshold, determine anotherplurality of tracking points from a next video frame.
 11. The mobilecomputer of claim 9, wherein determining the quality score for each ofthe tracking points, further comprises, determining an age of thetracking point and the variance between its predicted location and itsactual location within each video frame.
 12. The mobile computer ofclaim 9, wherein capturing the sequence of video frames of the videoscene using the mobile computer, further comprises, displaying a visualtiming cue in the mobile computer's view finder that indicates aduration for playing back the motion graphic.
 13. The mobile computer ofclaim 9, wherein displaying the virtual decal in the video scene,further comprises, modifying a location of the virtual decal within theview finder based on at least one dynamic property of the motiongraphic.
 14. The mobile computer of claim 9, further comprising,enabling a user to adjust a size of the virtual decal for determining asize of the motion graphic for embedding in each of the video frames.15. The mobile computer of claim 9, wherein determining a plurality oftracking points within the video frame, further comprises, computing ascore based on at least one of contrast, brightness, darkness, patternmatching, edge detection, or corner-recognition.
 16. The mobile computerof claim 9, further comprising, plotting a position of the motiongraphic using a coordinate system that is separate from anothercoordinate system that corresponds to each video frame.
 17. A processorreadable non-transitive storage media that includes instructions forvideo processing using a mobile computer, wherein the mobile computerthat executes at least a portion of the instructions enables actions,comprising: selecting a motion graphic for displaying at a location in avideo scene; employing a view finder provided by the mobile computer toview the video scene while positioning a virtual decal at the locationin the video scene, wherein the virtual decal represents the selectedmotion graphic; employing the mobile computer to capture a sequence ofvideo frames of the video scene while at least a portion of the virtualdecal is viewable in the video scene with the view finder; anditeratively processing each recorded video frame from the sequence ofcaptured video frames using the mobile computer, wherein the processingincludes: determining a plurality of tracking points within each videoframe based on at least one characteristic of an image included in thevideo frame; determining a quality score for each of the trackingpoints; if the quality score corresponding to an individual trackingpoint included in the plurality of tracking points is less than adefined threshold, discarding the individual tracking point from theplurality of tracking points; determining at least one reference pointwithin the video frame based on at least the location of a portion ofthe plurality of determined tracking points; and embedding the motiongraphic in the video frame at the location of the virtual decal, whereinthe motion graphic is further positioned in the video frame based atleast on the reference point.
 18. The media of claim 17, furthercomprising, if a quantity of tracking points included in the pluralityof tracking points drops below a threshold, determine another pluralityof tracking points from a next video frame.
 19. The media of claim 17,wherein determining the quality score for each of the tracking points,further comprises, determining an age of the tracking point and thevariance between its predicted location and its actual location withineach video frame.
 20. The media of claim 17, wherein capturing thesequence of video frames of the video scene using the mobile computer,further comprises, displaying a visual timing cue in the mobilecomputer's view finder that indicates a duration for playing back themotion graphic.
 21. The media of claim 17, wherein displaying thevirtual decal in the video scene, further comprises, modifying alocation of the virtual decal within the view finder based on at leastone dynamic property of the motion graphic.
 22. The media of claim 17,further comprising, enabling a user to adjust a size of the virtualdecal for determining a size of the motion graphic for embedding in eachof the video frames.
 23. The media of claim 17, wherein determining aplurality of tracking points within the video frame, further comprises,computing a score based on at least one of contrast, brightness,darkness, pattern matching, edge detection, or corner-recognition. 24.The media of claim 17, further comprising, plotting a position of themotion graphic using a coordinate system that is separate from anothercoordinate system that corresponds to each video frame.